What is the right course of action here?

Your choice is correct: you must let the yellow vehicle go first.

The key rule here is “Yield” (Vorfahrt gewähren). When you have a yield sign, you must give right of way to all traffic on the road you want to enter or cross, no matter whether they come from the left or the right. So even if you are turning, you cannot “take the turn ahead” of a vehicle that is already on the priority road.

Why the yellow vehicle goes first: it is traveling on the road with priority relative to you. You must wait until it has passed and the junction is clear enough to enter safely.

A good rule of thumb for similar questions: If you see a yield sign, think “everyone on the other road goes before me,” and only then do you consider who turns where. Turning direction doesn’t cancel the need to yield.